Q: Are we not starting randomization everywhere at the same time or do you have some villages identified already in e.g. West Africa?
A: This has not been fully decided yet, every region has started a slightly different approach.
Q: What is the difference between village 1 and village 2 (without intervention)?
A: The best way to look at this is to think about measuring the effects of fertilizers on yield across the 2 villages. We'd like to know the kinds of treatment effects (average response per Ha to use certain amounts of fertilizers. This is looking at spillover effects within and over a district. THis allows us to think about scaling the project and what would be the impact over other areas.
Comment: Looking at a larger picture, knowing how things work on the ground, picking another village means you need to look at the relationship of that village to markets etc. Maybe they're not applying fertilizers to the same crops etc. These need to be taken into account.
--> On village location, the idea is to come up with homogeneous strata looking at ag potential and market access so this design will be applied to specific strata.
Comment: I hope you will explore other methodologies than RCTs.
Q: Why do we have the hypothesis about demand-driven technology? What informs that.
A: This sort of program is working at household level and we want to explore these dimensions, under what influence etc. We have examples of supply-driven technology that is very successful but under very specific circumstances.
Q: To make this a bit more conceptually flexible, we need to look at other approaches for district level counter-factuals. In e.g. West Africa, between districts there's more variability than between villages. Finding pairing districts is going to be challenging. What we consider homogeneous districts are usually very different in terms of baseline richness etc. across only a distance of 20km you find dramatic differences in access to land etc. You need to build flexibility in the graph so that people don't think it's that easy to find equivalent districts.
Q: About intervention and non-intervention, how separated should the villages be?
A: The diagram is outdated. We know that multi-stakeholder platforms put in place influence and naturally those are what we want to define. There are high0intensity exposure areas. Village 2 doesn't get as much exposure if we focus on village 1.
--> We talked a lot (in the research framework workshop in Ibadan) about where should we need prescriptive or flexible. We have to tailor our methodologies around this. Methodologies are not prescriptive but principles are.
Q: Why do we call this an action-research project?
A: Because it's based on demand, adaptive in nature. Testing, validating technologies interactively with farmers qualifies it.
Q: There are going to be farmers that are not adopting and that is as important as looking at the counter-factual villages. We need to understand decision-making about what motivates adopters and non-adopters, not just surface responses. People adopt because they think they will get sthg else, not necessarily what you think. Farmers should be involved IN the monitoring and evaluation. What would they qualify as success and reasons for adoption?
--> (Gary) I need to understand collaborative testing vs. adoption - they are not the same. In the documents we hear about diffusion phase, adoption etc. and those are not like collaborative testing. When do we qualify adoption etc.?
--> If it doesn't look like what we originally planned it's not qualified as adoption but if it matters for farmers' innovations we should accept this.
Q: Stratifying randomized design - wasn't it aligned around hi-lo gradients on ag potential and market access? Within that we were supposed to pick clusters of villages.
Comment: You mentioned over-grazing and over-stocking, wouldn't you think we should talk about ??
Q: In the research framework we talk about demand-driven innovation but it doesn't square off with the selection sites (which is not demand-driven)... We will be selecting sites in Ethiopia in 10 days and we need to keep track of this.
Q: We don't look at admin districts. How do we square that with the selection of the sites if there's already a scale for platforms etc.?
A: ??
--> We need to use some kind of geographic area before we get on with the site selection process. Within those areas we need to choose the project sites, they can be districts. These distrcits are usually very homogeneous but within these districts you find a lot of variation. It's a convenient label but people shouldn't think they are homogeneous and their size varies a lot from country to country.
Q: The unit of the districts is very important for structuring what happens at villages (e.g. policies, money) - the districts are critical entities in the project.
Africa RISING M&E Expert Meeting
5-7 September 2012Large auditorium,ILRI Ethiopia, Addis Ababa
Back to the event home page
Plenary feedback on research design and M&E
Presentation by IFPRI on the research design:
Questions and answers:
Q: Are we not starting randomization everywhere at the same time or do you have some villages identified already in e.g. West Africa?
A: This has not been fully decided yet, every region has started a slightly different approach.
Q: What is the difference between village 1 and village 2 (without intervention)?
A: The best way to look at this is to think about measuring the effects of fertilizers on yield across the 2 villages. We'd like to know the kinds of treatment effects (average response per Ha to use certain amounts of fertilizers. This is looking at spillover effects within and over a district. THis allows us to think about scaling the project and what would be the impact over other areas.
Comment: Looking at a larger picture, knowing how things work on the ground, picking another village means you need to look at the relationship of that village to markets etc. Maybe they're not applying fertilizers to the same crops etc. These need to be taken into account.
--> On village location, the idea is to come up with homogeneous strata looking at ag potential and market access so this design will be applied to specific strata.
Comment: I hope you will explore other methodologies than RCTs.
Q: Why do we have the hypothesis about demand-driven technology? What informs that.
A: This sort of program is working at household level and we want to explore these dimensions, under what influence etc. We have examples of supply-driven technology that is very successful but under very specific circumstances.
Q: To make this a bit more conceptually flexible, we need to look at other approaches for district level counter-factuals. In e.g. West Africa, between districts there's more variability than between villages. Finding pairing districts is going to be challenging. What we consider homogeneous districts are usually very different in terms of baseline richness etc. across only a distance of 20km you find dramatic differences in access to land etc. You need to build flexibility in the graph so that people don't think it's that easy to find equivalent districts.
Q: About intervention and non-intervention, how separated should the villages be?
A: The diagram is outdated. We know that multi-stakeholder platforms put in place influence and naturally those are what we want to define. There are high0intensity exposure areas. Village 2 doesn't get as much exposure if we focus on village 1.
--> We talked a lot (in the research framework workshop in Ibadan) about where should we need prescriptive or flexible. We have to tailor our methodologies around this. Methodologies are not prescriptive but principles are.
Q: Why do we call this an action-research project?
A: Because it's based on demand, adaptive in nature. Testing, validating technologies interactively with farmers qualifies it.
Q: There are going to be farmers that are not adopting and that is as important as looking at the counter-factual villages. We need to understand decision-making about what motivates adopters and non-adopters, not just surface responses. People adopt because they think they will get sthg else, not necessarily what you think. Farmers should be involved IN the monitoring and evaluation. What would they qualify as success and reasons for adoption?
--> (Gary) I need to understand collaborative testing vs. adoption - they are not the same. In the documents we hear about diffusion phase, adoption etc. and those are not like collaborative testing. When do we qualify adoption etc.?
--> If it doesn't look like what we originally planned it's not qualified as adoption but if it matters for farmers' innovations we should accept this.
Q: Stratifying randomized design - wasn't it aligned around hi-lo gradients on ag potential and market access? Within that we were supposed to pick clusters of villages.
Comment: You mentioned over-grazing and over-stocking, wouldn't you think we should talk about ??
Q: In the research framework we talk about demand-driven innovation but it doesn't square off with the selection sites (which is not demand-driven)... We will be selecting sites in Ethiopia in 10 days and we need to keep track of this.
Q: We don't look at admin districts. How do we square that with the selection of the sites if there's already a scale for platforms etc.?
A: ??
--> We need to use some kind of geographic area before we get on with the site selection process. Within those areas we need to choose the project sites, they can be districts. These distrcits are usually very homogeneous but within these districts you find a lot of variation. It's a convenient label but people shouldn't think they are homogeneous and their size varies a lot from country to country.
Q: The unit of the districts is very important for structuring what happens at villages (e.g. policies, money) - the districts are critical entities in the project.